{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Numpy实现深层神经网络\n",
    "\n",
    "在该实验中我们将介绍如何使用Python及Numpy lib库实现深层神经网络模型来识别猫。本小节代码与第三章Pyhton版本代码大体一致，区别在于增加了3层隐藏层并设置不同节点数。\n",
    "\n",
    "** 图片处理 **\n",
    "\n",
    "由于识别猫问题涉及到图片处理指示，这里对计算机如何保存图片做一个简单的介绍。在计算机中，图片被存储为三个独立的矩阵，分别对应图3-6中的红、绿、蓝三个颜色通道，如果图片是64*64像素的，就会有三个64*64大小的矩阵，要把这些像素值放进一个特征向量中，需要定义一个特征向量X，将三个颜色通道中的所有像素值都列出来。如果图片是64*64大小的，那么特征向量X的总纬度就是64*64*3，也就是12288维。这样一个12288维矩阵就是Logistic回归模型的一个训练数据。\n",
    "\n",
    "<img src=\"images/image_to_vector.png\" style=\"width:550px;height:300px;\">\n",
    "\n",
    "## 1 - 引用库\n",
    "\n",
    "首先，载入几个需要用到的库，它们分别是：\n",
    "- numpy：一个python的基本库，用于科学计算\n",
    "- matplotlib.pyplot：用于生成图，在验证模型准确率和展示成本变化趋势时会使用到\n",
    "- h5py：用于处理hdf5文件数据\n",
    "- dnn_app_utils_v2：包含了一些有用的工具函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import h5py\n",
    "import dnn_app_utils_v2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2 - 数据预处理\n",
    "\n",
    "这里简单介绍数据集及其结构。数据集以hdf5文件的形式存储，包含了如下内容：\n",
    "\n",
    "- 训练数据集：包含了m_train个图片的数据集，数据的标签（Label）分为cat（y=1）和non-cat（y=0）两类。\n",
    "- 测试数据集：包含了m_test个图片的数据集，数据的标签（Label）同上。\n",
    "\n",
    "单个图片数据的存储形式为（num_x, num_x, 3），其中num_x表示图片的长或宽（数据集图片的长和宽相同），数字3表示图片的三通道（RGB）。\n",
    "在代码中使用一行代码来读取数据，读者暂不需要了解数据的读取过程，只需调用load_dataset()方法，并存储五个返回值，以便后续的使用。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "train_x_orig, train_y, test_x_orig, test_y, classes = dnn_app_utils_v2.load_data() \n",
    "#数据预处理 \n",
    "train_x_flatten = train_x_orig.reshape(train_x_orig.shape[0], -1).T   # The \"-1\" makes reshape flatten the remaining dimensions\n",
    "test_x_flatten = test_x_orig.reshape(test_x_orig.shape[0], -1).T\n",
    "\n",
    "#归一化\n",
    "train_x = train_x_flatten/255.\n",
    "test_x = test_x_flatten/255."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3 - 建立神经网络模型\n",
    "\n",
    "对比“浅层神经网络”的网络结构，本小节神经网络模型有以下不同：\n",
    "\n",
    "- 增加2层隐藏层，共有3层隐藏层和1层输出层；\n",
    "\n",
    "- 第一层设置20个节点，第二层7个，第三层5个；\n",
    "\n",
    "- 隐藏层激活函数使用Relu激活函数。\n",
    "\n",
    "在dnn_app_utils_v2文件中已包含下列函数，在实现神经网络模型中将直接调用（下列函数在第四章Python代码部分均有实现，根据网络结构不同略有差异）：\n",
    "\n",
    "- 初始化参数\n",
    "\n",
    "initialize_parameters_deep(layer_dims)\n",
    "\n",
    "\n",
    "- 正向传播\n",
    "\n",
    "L_model_forward(X, parameters)\n",
    "\n",
    "\n",
    "- 成本函数\n",
    "\n",
    "compute_cost(AL, Y)\n",
    "\n",
    "\n",
    "- 反向传播\n",
    "\n",
    "L_model_backward(AL, Y, caches)\n",
    "\n",
    "\n",
    "- 参数更新\n",
    "\n",
    "update_parameters(parameters, grads, learning_rate)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#设置神经网络规模，5个数字分别表示从输入层到隐藏层到输出层各层节点数\n",
    "layers_dims = [12288, 20, 7, 5, 1] \n",
    " \n",
    "#定义函数：L层神经网络模型\n",
    "def L_layer_model(X, Y, layers_dims, learning_rate = 0.0075, num_iterations = 3000, print_cost=False):\n",
    "    \"\"\"\n",
    "    L层神经网络正向传播: [加权偏移->RELU激活]*(L-1)->加权激活->SIGMOID激活.\n",
    "    \n",
    "    参数:\n",
    "    X -- 输入值，维度为 (样本个数, 像素num_px * num_px * 3)\n",
    "    Y -- 真实值（0表示不是猫，1表示是猫），维度为 (1, 样本个数)\n",
    "    layers_dims -- 各层节点数\n",
    "    learning_rate -- 学习率\n",
    "    num_iterations -- 训练次数\n",
    "    print_cost -- 参数设置True，则每100次训练打印一次成本函数值\n",
    "    \n",
    "    返回值:\n",
    "    parameters -- 模型训练所得参数，用于预测\n",
    "    \"\"\"\n",
    "\n",
    "    np.random.seed(1)\n",
    "    costs = []                                                                \n",
    "\n",
    "    #参数初始化\n",
    "    parameters = dnn_app_utils_v2.initialize_parameters_deep(layers_dims)\n",
    "    \n",
    "    #训练\n",
    "    for i in range(0, num_iterations):\n",
    "\n",
    "        #正向传播: [加权偏移 -> RELU激活]*(L-1) -> 加权偏移后 -> SIGMOID激活       \n",
    "        AL, caches = dnn_app_utils_v2.L_model_forward(X, parameters)\n",
    "        \n",
    "        #计算成本函数\n",
    "        cost = dnn_app_utils_v2.compute_cost(AL, Y)\n",
    "    \n",
    "        #反向传播\n",
    "        grads = dnn_app_utils_v2.L_model_backward(AL, Y, caches)\n",
    " \n",
    "        #更新参数\n",
    "        parameters = dnn_app_utils_v2.update_parameters(parameters, grads, learning_rate)\n",
    "                \n",
    "        #每100次训练打印一次成本函数\n",
    "        if print_cost and i % 100 == 0:\n",
    "            print (\"Cost after iteration %i: %f\" %(i, cost))\n",
    "        if print_cost and i % 100 == 0:\n",
    "            costs.append(cost)\n",
    "            \n",
    "    #绘制损失函数变化折线图\n",
    "    plt.plot(np.squeeze(costs))\n",
    "    plt.ylabel('cost')\n",
    "    plt.xlabel('iterations (per tens)')\n",
    "    plt.title(\"Learning rate =\" + str(learning_rate))\n",
    "    plt.show()\n",
    "    \n",
    "    return parameters\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面开始训练，训练2500次，观察成本函数变化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cost after iteration 0: 0.771749\n",
      "Cost after iteration 100: 0.672053\n",
      "Cost after iteration 200: 0.648263\n",
      "Cost after iteration 300: 0.611507\n",
      "Cost after iteration 400: 0.567047\n",
      "Cost after iteration 500: 0.540138\n",
      "Cost after iteration 600: 0.527930\n",
      "Cost after iteration 700: 0.465477\n",
      "Cost after iteration 800: 0.369126\n",
      "Cost after iteration 900: 0.391747\n",
      "Cost after iteration 1000: 0.315187\n",
      "Cost after iteration 1100: 0.272700\n",
      "Cost after iteration 1200: 0.237419\n",
      "Cost after iteration 1300: 0.199601\n",
      "Cost after iteration 1400: 0.189263\n",
      "Cost after iteration 1500: 0.161189\n",
      "Cost after iteration 1600: 0.148214\n",
      "Cost after iteration 1700: 0.137775\n",
      "Cost after iteration 1800: 0.129740\n",
      "Cost after iteration 1900: 0.121225\n",
      "Cost after iteration 2000: 0.113821\n",
      "Cost after iteration 2100: 0.107839\n",
      "Cost after iteration 2200: 0.102855\n",
      "Cost after iteration 2300: 0.100897\n",
      "Cost after iteration 2400: 0.092878\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYYAAAEWCAYAAABi5jCmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VOXZ//HPlYVAyMKSEPYdZFHcAiiLWrVVq9W6VrDu\nSmlrN/vrU/s8Xexjt6fVVlt33KrWrdVatFrqUhVBkKDIKhjZ97BvgZDk+v0xBzpJk5BATk4m832/\nXvNi5sw9Z66bgfnOOfc59zF3R0RE5ICUqAsQEZHmRcEgIiJVKBhERKQKBYOIiFShYBARkSoUDCIi\nUoWCQVoEM3vVzK6Oug6RlkDBIEfEzJab2ZlR1+Hu57j7H6OuA8DM3jKzG5rgfTLM7BEz22Fm683s\n5kO0H29mK8xst5m9aGYd6rMuMxtrZruq3dzMLg6ev8bMKqo9f1poHZfQKRik2TOztKhrOKA51QLc\nCgwAegGfAf7LzM6uqaGZDQUeAK4ECoA9wL31WZe7T3X3rAM34DxgF/CPuNe/F9/G3d9qtF5Kk1Mw\nSGjM7Dwzm2Nm28xsupkNi3vuFjP71Mx2mtlCM7sw7rlrzGyamf3OzDYDtwbL3jWz281sq5ktM7Nz\n4l5z8Fd6Pdr2MbN3gvd+3czuMbMna+nDaWa22sy+b2brgUfNrL2ZvWxmJcH6Xzaz7kH7nwNjgbuD\nX853B8sHmdlrZrbFzBab2WWN8Fd8NXCbu29190XAg8A1tbS9AnjJ3d9x913Aj4CLzCz7MNZ1NfAX\nd9/dCH2QZkjBIKEws+OBR4CvAB2J/VqdbGYZQZNPiX2B5gI/BZ40sy5xqxgJLCX26/bnccsWA3nA\nr4GHzcxqKaGutk8B7wd13UrsV3RdOgMdiP2ankDs/82jweOeQClwN4C7/w8wFbgp+OV8k5m1BV4L\n3rcTcDlwr5kNqenNzOzeIExrus0N2rQHugAfxb30I2BoLX0YGt/W3T8F9gEDG7KuoC+XANV32x1v\nZpvMbImZ/aiZbVlJAykYJCwTgAfcfaa7VwT7//cBJwG4+5/dfa27V7r7s8AnwIi416919z+4e7m7\nlwbLVrj7JHevIPbF1IVYcNSkxrZm1hMYDvzY3cvc/V1g8iH6Ugn8xN33uXupu2929+fdfY+77yQW\nXKfW8frzgOXu/mjQnw+B54FLa2rs7l9z93a13A5sdWUFf26Pe+kOIJuaZVVrG9++Ieu6CNgEvB23\n7B3gaGKhdzEwDvheLXVIAlAwSFh6Ad+N/7UL9AC6ApjZVXG7mbYR+2LJi3v9qhrWuf7AHXffE9zN\nqqFdXW27AlviltX2XvFK3H3vgQdmlmlmDwQDuTuIfTG2M7PUWl7fCxhZ7e/iCmJbIodrV/BnTtyy\nXGBnHe1zqi070L4h67oaeNzjZt9096XuviwI+XnA/xLbqpAEpWCQsKwCfl7t126muz9tZr2AScBN\nQEd3bwfMB+J3C4U17e86oIOZZcYt63GI11Sv5bvAUcBId88BTgmWWy3tVwFvV/u7yHL3r9b0ZmZ2\nfw1HAR24LQBw961BX46Ne+mxwIJa+rAgvq2Z9QNaAUvquy4z6wGcBjxey3sc4FT9LCXBKBikMaSb\nWeu4WxqxL/6JZjbSYtqa2bnBYGdbYl8eJQBmdi2xLYbQufsKoIjYgHYrMzsZ+EIDV5NNbFxhm8UO\n+fxJtec3AH3jHr9MbF/+lWaWHtyGm9ngWmqcWO0In/hb/H7/x4EfBoPhg4EbgcdqqflPwBcsduhp\nW+A24IVgV1h913UlMD0YnzjIzM4xs4Lg/iBiA9t/q6UOSQAKBmkMrxD7ojxwu9Xdi4h9udwNbAWK\nCY5ycfeFwB3Ae8S+RI8BpjVhvVcAJwObgZ8BzxIb/6ivO4E2xPa1z6DqYZsAdwGXBEcs/T748v0c\nsUHntcR2c/0fkMGR+QmxQfwVwFvAr939YC3BFsZYAHdfAEwkFhAbiYXz1+q7rsBV/OegM8AZwFwz\n203s38ILwC+OsG8SIdOFeiTZmdmzwMfuXv2Xv0hS0haDJJ1gN04/M0ux2ElcFwAvRl2XSHOhY40l\nGXUmtrujI7Aa+GpwCKmIoF1JIiJSTai7kszs7OD0/2Izu6WG53PN7CUz+8jMFgRHp4iISIRC22II\nTvZZAnyW2Ob6LGBccETKgTb/DeS6+/fNLJ/YFAad3b2stvXm5eV57969Q6lZRKSlmj179iZ3z69P\n2zDHGEYAxe6+FMDMniE2yLcwro0D2cEcNlnAFqC8rpX27t2boqKicCoWEWmhzGxFfduGuSupG1Wn\nGlgdLIt3NzCY2LHd84BvuXtl9RWZ2QQzKzKzopKSkrDqFRERoj9c9SxgDrH5a44jNlVx9flccPcH\n3b3Q3Qvz8+u1JSQiIocpzGBYQ9U5aLoHy+JdS+y0fHf3YmAZMCjEmkRE5BDCDIZZwACLXRSlFbHp\nAKpPb7yS2On0BHOtHEVsDn4REYlIaIPP7l5uZjcBU4BU4BF3X2BmE4Pn7yc2kddjZjaP2GyM33f3\nTWHVJCIihxbqmc/u/gqxSbXil90fd38tscnFRESkmYh68FlERJqZpAmGT0t28dOXFrC/4j+OhhUR\nkThJEwwrN+/h0WnLeWXeuqhLERFp1pImGE4dmE/fvLY8Om151KWIiDRrSRMMKSnGNaN7M2fVNj5Y\nuTXqckREmq2kCQaAi0/oTnbrNB55d1nUpYiINFtJFQxtM9IYN6Inr85fz9ptpVGXIyLSLCVVMABc\ndXIv3J3H36v3RIMiIkkl6YKhe/tMzj66M0+/v5I9ZXXO8C0ikpSSLhgArh3dh+2l+/nrh9Xn9BMR\nkaQMhsJe7TmmWy6PvLuMykpd81pEJF5SBoOZcd2Y3nxaspupxZqzT0QkXlIGA8C5x3QlPztDh66K\niFSTtMHQKi2Fq07qxdtLSijeuDPqckREmo2kDQaA8SN70iotRdNkiIjESepg6JiVwYXHdeOFD9aw\nbU9Z1OWIiDQLSR0MANeO6U3p/gqembUq6lJERJqFpA+GQZ1zGNWvI3+cvlzXahARQcEAwHWj+7Bu\n+16mLFgfdSkiIpFTMACnD+pEr46ZOnRVRISQg8HMzjazxWZWbGa31PD898xsTnCbb2YVZtYhzJpq\nkpJiXDuqNx+s3MacVdua+u1FRJqV0ILBzFKBe4BzgCHAODMbEt/G3X/j7se5+3HAD4C33X1LWDXV\n5ZLCHmRnpPHoNG01iEhyC3OLYQRQ7O5L3b0MeAa4oI7244CnQ6ynTlkZaVw2vAd/n7uO9dv3RlWG\niEjkwgyGbkD8MaCrg2X/wcwygbOB52t5foKZFZlZUUlJSaMXesA1o3pT6c4TM5aH9h4iIs1dcxl8\n/gIwrbbdSO7+oLsXunthfn5+aEX06JDJZ4cU8NTMlZSWVYT2PiIizVmYwbAG6BH3uHuwrCaXE+Fu\npHjXje7D1j37eXGOrtUgIskpzGCYBQwwsz5m1orYl//k6o3MLBc4FfhbiLXU24g+HRjaNYdHpy3D\nXddqEJHkE1owuHs5cBMwBVgEPOfuC8xsoplNjGt6IfBPd98dVi0NYWZcN7oPSzbsYlrx5qjLERFp\ncpZov4oLCwu9qKgo1PfYV17B6F/9i2Hdc3nkmuGhvpeISFMws9nuXlifts1l8LlZyUhL5csn9eTN\njzeytGRX1OWIiDQpBUMtrhjZi1apKdz71qe6LrSIJBUFQy3yszMYN6IHf5m9mvP+8C5TPwnv/AkR\nkeZEwVCHn3xhKL8fdzw79+3nyoff58qHZ7Jg7faoyxIRCZWCoQ4pKcb5x3bl9ZtP5UfnDWHemu2c\n94d3ufnZOazeuifq8kREQqGjkhpge+l+7n/7Ux55dxkOXDuqN187rT+5memR1CMiUl8NOSpJwXAY\n1m4r5bevLeH5D1aT0zqdb5zenytP7kVGWmqkdYmI1EaHq4asa7s23H7psbzyzbEc16MdP/v7Is64\n421e/HCNjmASkYSnYDgCg7vk8MfrRvDk9SPJbZPOt5+dwxfufpe5q3WxHxFJXAqGRjBmQB4v3TSG\nuy4/jq27y7j0/vd4ee7aqMsSETksCoZGkpJiXHBcN176xhiGdc/lpqc+5PdvfKKJ+EQk4SgYGlnH\nrAyevGEkF53Qjd++toTvPDuHvft1bQcRSRxpURfQEmWkpXLHpcfSLz+L30xZzMote3jwqkLysjKi\nLk1E5JC0xRASM+Prn+nPvVecwMJ1O/jiPdNYvH5n1GWJiBySgiFknz+mC8995WTKyiu5+L7p/Gvx\nxqhLEhGpk4KhCQzr3o6/3TSanh0yuf6xWTw2bVnUJYmI1ErB0ES65LbhzxNP5vRBBdz60kJ+9OJ8\nyisqoy5LROQ/KBiaUNuMNB648kS+ckpfnpixgmsfm8X20v1RlyUiUoWCoYmlphg/+Pxg/u/iY3jv\n081cfN90Vm7WTK0i0nwoGCLypeE9eeL6kZTs3McX753G/DW6zoOINA+hBoOZnW1mi82s2MxuqaXN\naWY2x8wWmNnbYdbT3JzcryMvfn00bdJTufLhmXy8fkfUJYmIhBcMZpYK3AOcAwwBxpnZkGpt2gH3\nAue7+1Dg0rDqaa765LXlqRtHkpGWyhWTZvLJBp3rICLRCnOLYQRQ7O5L3b0MeAa4oFqb8cAL7r4S\nwN2T8iD/Xh1j4ZCaYoybNJPijbuiLklEkliYwdANWBX3eHWwLN5AoL2ZvWVms83sqppWZGYTzKzI\nzIpKSkpCKjdaffOzeOrGkwBn/KQZLNu0O+qSRCRJRT34nAacCJwLnAX8yMwGVm/k7g+6e6G7F+bn\n5zd1jU2mf6dYOJRXxsJBRyuJSBTCDIY1QI+4x92DZfFWA1Pcfbe7bwLeAY4NsaZmb2BBNk9eP5LS\n/RWMmzSD1VsVDiLStMIMhlnAADPrY2atgMuBydXa/A0YY2ZpZpYJjAQWhVhTQhjSNYcnrx/Jzr37\nGTdpBmu3lUZdkogkkdCCwd3LgZuAKcS+7J9z9wVmNtHMJgZtFgH/AOYC7wMPufv8sGpKJEd3y+WJ\n60eybfd+xk+awfrte6MuSUSShCXaFcYKCwu9qKgo6jKazOwVW7nq4ZkU5LbmmQkn0Sm7ddQliUgC\nMrPZ7l5Yn7ZRDz7LIZzYqz2PXTeCddv2csWkmWzatS/qkkSkhVMwJIDhvTvwyDXDWbV1D19+aCZb\ndpdFXZKItGAKhgRxcr+OPHTVcJZt2s2XH5rJtj0KBxEJh4IhgYwZkMcDV55I8cZdfOmBGcxZtS3q\nkkSkBVIwJJjTjurEQ1cXsnVPGRfeO40f/20+O/bqmg4i0ngUDAnolIH5vPHdU7n65N48OWMFZ9zx\nNpM/WkuiHWEmIs2TgiFBZbdO59bzh/Li10fTOac133z6Q6565H2Wa44lETlCCoYEN6x7O178+mh+\nev5QPly5jc/d+Q53vf4J+8oroi5NRBKUgqEFSE0xrh7Vmze+eyqfG1LA715fwjl3TmV68aaoSxOR\nBKRgaEEKclpz9/gT+ON1I6hwZ/xDM/nOs3N0UpyINIiCoQU6dWA+U759Ct88vT8vz13L6be/xVMz\nV1JZqcFpETk0zZXUwhVv3MWPXpzPe0s3k5+dwdgBeZw6MJ+xA/Lp0LZV1OWJSBNpyFxJCoYk4O78\nY/56Xp2/nqmflLB1z37MYgPXpw7I49Sj8jm2ezvSUrUBKdJSKRikVhWVzrw123l7cQnvfFLChyu3\nUumQ0zqNsQPyOWVgHqcMzKdLbpuoSxWRRqRgkHrbvmc/7xZv4u0lG3lnySbW74hd9+GogmxOH9yJ\nb5zen8xWaRFXKSJHqiHBoP/xSS43M51zh3Xh3GFdcHeWbNh1MCTuf/tTtu4u41cXD4u6TBFpQtqp\nLAeZGUd1zmbCKf148oaRTDy1H8/MWsVrCzdEXZqINCEFg9TqO2cOZEiXHG55fi4lO3UuhEiyUDBI\nrVqlpXDn5cexc185tzw/V5P0iSQJBYPUaWBBNt8/exBvfLyRZ2atirocEWkCoQaDmZ1tZovNrNjM\nbqnh+dPMbLuZzQluPw6zHjk8147qzej+Hbnt5YWavVUkCYQWDGaWCtwDnAMMAcaZ2ZAamk519+OC\n2/+GVY8cvpQU4/ZLjyUtxfjOc3Mor6iMuiQRCVGYWwwjgGJ3X+ruZcAzwAUhvp+EqEtuG3524TF8\nuHIb9731adTliEiIwgyGbkD8TunVwbLqRpnZXDN71cyG1rQiM5tgZkVmVlRSUhJGrVIP5x/blfOP\n7cpdb3zC3NW63rRISxX14PMHQE93Hwb8AXixpkbu/qC7F7p7YX5+fpMWKFXddsHR5Gdn8O1n51Ba\nposBibREYQbDGqBH3OPuwbKD3H2Hu+8K7r8CpJtZXog1yRHKzUzn9kuPZWnJbn756qKoyxGREIQZ\nDLOAAWbWx8xaAZcDk+MbmFlnM7Pg/oigns0h1iSNYHT/PK4f04fH31vBW4s3Rl2OiDSy0ILB3cuB\nm4ApwCLgOXdfYGYTzWxi0OwSYL6ZfQT8HrjcdRZVQvjeWUcxsCCL//rLXLbuLou6HBFpRJpdVQ7b\ngrXb+eI90zhzcAH3XnECwcafiDRDDZldNerBZ0lgQ7vmcvNnj+LV+et54YM1h36BiCQEBYMckQmn\n9GVE7w78ZPICVm3ZE3U5ItIIFAxyRFJTjDsuOxaA7/75IyoqE2vXpIj8JwWDHLEeHTK59fyhvL9s\nC5OmLo26HBE5QgoGaRQXn9CNs4YW8LvXlrBpl67dIJLIFAzSKMyM7501iH3llTzx3oqoyxGRI1Cv\nYDCzS+uzTJJb/05ZnDGoE0/MWKHpMkQSWH23GH5Qz2WS5G48pS9bdpfx/Aeroy5FRA5TWl1Pmtk5\nwOeBbmb2+7incoDyMAuTxDSyTweGdc/l4XeXMX5ET1JSdNKbSKI51BbDWqAI2AvMjrtNBs4KtzRJ\nRGbGjWP7smzTbl5ftCHqckTkMNS5xeDuHwEfmdlT7r4fwMzaAz3cfWtTFCiJ55yjO9OtXRsmTV3K\n54Z2jrocEWmg+o4xvGZmOWbWgdg1FCaZ2e9CrEsSWFpqCteN6cOs5Vv5cKV+P4gkmvoGQ6677wAu\nAh5395HAGeGVJYnuS8N7kN06jYemLou6FBFpoPoGQ5qZdQEuA14OsR5pIbIy0hg/sievzl+nOZRE\nEkx9g+F/iV1X4VN3n2VmfYFPwitLWoJrR/UhxYyH39VWg0giqVcwuPuf3X2Yu381eLzU3S8OtzRJ\ndJ1zW3P+cV15rmgV2/boYj4iiaK+Zz53N7O/mtnG4Pa8mXUPuzhJfDeO7cuesgr+NHNl1KWISD3V\nd1fSo8TOXega3F4KlonUaXCXHMYOyOOx6cvZV65pMkQSQX2DId/dH3X38uD2GJAfYl3Sgtw4ti8l\nO/cxec7aqEsRkXqobzBsNrMvm1lqcPsysDnMwqTlGDsgj0Gds5k0dSmJdo1xkWRU32C4jtihquuB\ndcAlwDWHepGZnW1mi82s2MxuqaPdcDMrN7NL6lmPJBAz44axfVmyYRdvLymJuhwROYSGHK56tbvn\nu3snYkHx07peYGapwD3AOcAQYJyZDaml3f8B/2xI4ZJYzj+2KwU5GTrhTSQB1DcYhsXPjeTuW4Dj\nD/GaEUBxcGhrGfAMcEEN7b4BPA9srGctkoBapaVwzag+vFu8iQVrt0ddjojUob7BkBJMngdAMGdS\nnRPwAd2AVXGPVwfLDjKzbsCFwH11rcjMJphZkZkVlZRoV0SiGj+yJ21bpWqrQaSZq28w3AG8Z2a3\nmdltwHTg143w/ncC33f3yroaufuD7l7o7oX5+ToYKlHltknnsuE9eOmjtazbXhp1OSJSi/qe+fw4\nsQn0NgS3i9z9iUO8bA3QI+5x92BZvELgGTNbTmxA+14z+2J9apLEdN3oPlS689i05Q1+bVl5JU/M\nWMHF901n4dodjV+ciACH3h10kLsvBBY2YN2zgAFm1odYIFwOjK+2zj4H7pvZY8DL7v5iA95DEkyP\nDpl8/pguPDVzJTed3p/s1umHfE1FpfPXD9dw5+tLWL21lBSDn0yez3NfORkzXSFOpLHVd1dSg7l7\nOXATscn3FgHPufsCM5toZhPDel9p/m4c25ed+8p5dtaqOttVVjp/n7uOz/3ubf7fnz+iXWY6j147\nnNu+eDSzlm/llXnrm6hikeRiiXbCUWFhoRcVFUVdhhyhyx54jzVbS3nre6eRnlr194m786/FG7l9\nyhIWrtvBgE5Z3PzZgZx9dGfMjIpK59zfT2XXvnJev/lUWqenRtQLkcRhZrPdvbA+bUPbYhCpy4Sx\nfVmzrZRX5q2rsnz6p5u4+L7pXPdYEbv2lfO7Lx3LP759Cucc0+XgbqPUFONH5w1h9dZSHpmmI5xE\nGlu9xxhEGtPpgzrRN78tk6Yu5fxjuzJn1TZu/+diphVvpnNOa35x4TFcWtj9P7YmDhjdP48zBxdw\nz5vFXHJidzplt27iHoi0XNpikEikpBg3jOnL/DU7uPT+97jw3ul8vG4nPzpvCG997zTGj+xZaygc\n8D/nDqasopI7pixpoqpFkoOCQSJz0Qnd6JSdweINO/l/nxvIO//1Ga4f06feYwZ98tpy9cm9eW72\nKp1NLdKINPgskdq0ax8ZaSn1Omy1JttL93Pab/7FUZ2zefrGk3T4qkgtNPgsCSMvK+OwQwFiZ1Pf\n/NmBzFi6hX8u3NCIlYkkLwWDJLxxI3oyoFMWv3hlka4SJ9IIFAyS8NJSU/jheUNYsXkPf5y+POpy\nRBKegkFahFMH5vOZo/L5wxvFbN61L+pyRBKagkFajP85dwh79lfw29d0+KrIkVAwSIvRv1MWV57U\ni6ffX8ni9TujLkckYSkYpEX51hkDyG6dzm0vLyTRDsUWaS4UDNKitG/bim+fOYB3izfx5se6WqzI\n4VAwSIvz5ZN60Te/LT//+yL2V9R5cUARqYGCQVqc9NQUfnjuYJZu2s0T762IuhyRhKNgkBbpM0d1\nYuyAPO564xO27i6LuhyRhKJgkBbJzPjhuUPYuXc/d73xSdTliCQUBYO0WEd1zmb8yJ48MWMFxRt1\n+KpIfSkYpEX7zpkDyWyVyv++vIjKSh2+KlIfCgZp0TpmZfC9s47inSUlfO8vc6lQOIgcUqjBYGZn\nm9liMys2s1tqeP4CM5trZnPMrMjMxoRZjySnK0/qxXfOHMjzH6zm5ufmUK5DWEXqFNo1n80sFbgH\n+CywGphlZpPdfWFcszeAye7uZjYMeA4YFFZNkpzMjG+dOYC0VOM3UxZTXuHceflxh7x0qEiyCi0Y\ngBFAsbsvBTCzZ4ALgIPB4O674tq3BbSdL6H5+mf6k5GWws/+voiyikruHn88GWn1u4yoSDIJ8ydT\nN2BV3OPVwbIqzOxCM/sY+DtwXYj1iHDD2L789PyhvLZwA1998gP27teFfUSqi3xb2t3/6u6DgC8C\nt9XUxswmBGMQRSUlJU1boLQ4V4/qzS8uPIY3P97IjY8XUVqmcBCJF2YwrAF6xD3uHiyrkbu/A/Q1\ns7wannvQ3QvdvTA/P7/xK5WkM35kT359yTDeLd7EdY/NYk9ZedQliTQbYQbDLGCAmfUxs1bA5cDk\n+AZm1t/MLLh/ApABbA6xJpGDLivswW8vO5aZyzZzzSOz2LVP4SACIQaDu5cDNwFTgEXAc+6+wMwm\nmtnEoNnFwHwzm0PsCKYvuSbRlyZ04fHd+f2445m9citXPjyTHXv3R12SSOQs0b6HCwsLvaioKOoy\npIX5x/z1fOPpDxjcJYfHrxtBu8xWUZck0qjMbLa7F9anbeSDzyLNwdlHd+b+L5/Ix+t2Mn7STLZo\nRlZJYgoGkcAZgwuYdHUhn5bsYtyDMyjZuS/qkkQioWAQiXPqwHweuWY4K7bs5qL7pvHx+h1RlyTS\n5BQMItWM7p/H0zeexL79lVx073Renbcu6pJEmpSCQaQGx/dsz0vfGMPAgmy++qcP+O0/F2vabkka\nCgaRWhTktOaZCSdx6Ynd+f2bxUx4YjY7dTirJAEFg0gdWqen8utLhnHrF4bwr8UbufDe6SzbtDvq\nskRCpWAQOQQz45rRfXji+hFs3rWPC+5+l7cWb4y6LJHQKBhE6mlUvzwm3zSGru3acN1js3jg7U9J\ntBNERepDwSDSAD06ZPLC10ZxztFd+OWrH/OtZ+ZodlZpcRQMIg2U2SqNu8cfz/fOOoqX5q7lkvun\ns2ZbadRliTQaBYPIYTAzvv6Z/jx8dSErN+/h/D+8y/vLtkRdlkijUDCIHIHTBxXw16+PJrdNOuMn\nzeDZWSujLknkiCkYRI5Q/05ZvHjTaE7u15HvPz+P26cs1qC0JDQFg0gjyGmdziPXDOfy4T24+1/F\nfPvZOewr16C0JKa0qAsQaSnSU1P45UXH0KNDJr+Zsph12/fy4JUn6toOknC0xSDSiA4MSt91+XHM\nWbmNi+6bzsrNe6IuS6RBFAwiIbjguG48ecNINu8q48J7p/Hhyq1RlyRSbwoGkZCM6NOBF742irYZ\naYybNIN/zF8fdUki9aJgEAlRv/wsXvjaKAZ1zuGrf5rNw+8ui7okkUNSMIiELC8rg2cmnMRZQzpz\n28sLuXXyAip0bQdpxkINBjM728wWm1mxmd1Sw/NXmNlcM5tnZtPN7Ngw6xGJSuv0VO654gRuGNOH\nx6Yv5ytPzGZPWXnUZYnUKLRgMLNU4B7gHGAIMM7MhlRrtgw41d2PAW4DHgyrHpGopaYYPzxvCD89\nfyhvfryByx+cwcade6MuS+Q/hHkewwig2N2XApjZM8AFwMIDDdx9elz7GUD3EOsRaRauHtWbbu3a\n8I2nP+TsO6cydkAew3t3YESfDvTPzyIlxaIuUZJcmMHQDVgV93g1MLKO9tcDr9b0hJlNACYA9OzZ\ns7HqE4nMmUMK+PPEk7nvrU+Z/ulm/jZnLQDtMtMp7NWe4b07MLxPB47umkurNA0FStNqFmc+m9ln\niAXDmJqed/cHCXYzFRYWatROWoSju+VyzxUn4O6s3LKH95dtYdbyLcxavpXXF8WuENc6PYXjerRj\nRBAUJ/SJWydOAAAOr0lEQVRsT9uMZvHfVlqwMP+FrQF6xD3uHiyrwsyGAQ8B57j75hDrEWmWzIxe\nHdvSq2NbLi2M/Zcp2bmPouVbeH95LCzu/lcxlW/GxinOHNyJW88fSpfcNhFXLi2VhTULpJmlAUuA\nM4gFwixgvLsviGvTE3gTuKraeEOtCgsLvaioKISKRZqvXfvK+WDFVqZ9uok/Tl9OWkoKt5wziPEj\nempMQurFzGa7e2G92oY5PbCZfR64E0gFHnH3n5vZRAB3v9/MHgIuBlYELyk/VOEKBkl2Kzfv4Qd/\nncu04s2M6N2BX158DP3ys6IuS5q5ZhMMYVAwiIC78+fZq/nZywvZW17Jt84YwIRT+pKeqoFqqVlD\ngkH/ikQSkJlxWWEPXv/uqZw5uBO/mbKY8++exrzV26MuTVoABYNIAuuU3Zp7rziR+798Ipt37eOC\ne97ll68sorRMFwmSw6dgEGkBzj66M6/dfCpfGt6DB95Zytl3vcP04k1RlyUJSsEg0kLktknnlxcN\n46kbR2LA+Idm8v2/zGV76f6oS5MEo8FnkRZo7/4Kfvf6Eh6auowObVsxbngPzhhcwDHdcnV4a5LS\nUUkiAsD8Ndv52d8X8v6yLVR6bArw0wflc8bgAsb0z9NZ1ElEwSAiVWzZXcbbSzby+qKNvLO4hJ37\nymmVmsJJ/TpyxqBOnD6oEz06ZEZdpoRIwSAitdpfUcms5Vt4c9FG3vh4I8s27QbgqIJsTh/ciTMH\nd+K4Hu1J1S6nFkXBICL1trRkF29+vJE3Fm1k1vItlFd6MMtrB0b2iU3eN7Rrjk6eS3ANCQbtYBRJ\ncn3zs+ibn8UNY/uyvXQ/Uz8p4e3FJcxavoXXF20AILNVKif0bH/wuhHH92xH6/TUiCuXsGiLQURq\ntXHH3tgMr8u28P7yrXy8fgfukJ5qHNMtlxF9OjKiT3tO7NWB3DbpUZcrddCuJBEJxfbS/cxesYWZ\ny2JhMW/NdvZXOGbQPz+LwV1yGNwlhyFdcxjcJZtO2a2jLlkC2pUkIqHIbZPO6YMKOH1QAQClZRV8\nuGors5ZtZd6abRQt38Lkj9YebJ+X1SoWFEFgDO6SQ9/8thqvaOYUDCJy2Nq0SmVUvzxG9cs7uGzb\nnjIWrdvJonU7WLRuBwvX7eDRacspq6gEoFVqCgMKshjSJYehXXM4ulsug7vk6JyKZkS7kkQkdPsr\nKllasrtKWCxat4NNu8oAMIO+eW05ulsux3TLZWjXXIZ0zdG4RSPSGIOINHvuzsad+5i/Zjvz1+xg\n/trtLFiznbXb9x5s06tjJkd3zWVot5zYn11z6JiVEWHViUtjDCLS7JkZBTmtKchpzRmDCw4u37xr\nHwvWxoJi/prtzFuznb/PW3fw+bysDAYWZDGwIJuBBdkc1TmLAQXZ5LTW1kVjUTCISLPSMSuDUwbm\nc8rA/IPLtu/Zz4J121m4dgdLNuxkyYZd/LloFbvjrjvRJbc1AwqyOaogK/gzmwEFWWS20tdcQ+lv\nTESavdzM9P8Y5K6sdNZuLz0YFEvW72TJxp08/t5m9pVXHmzXrV0benbIpFfHTHp2zKRXh7b06hh7\nnK2tjBopGEQkIaWkGN3bZ9K9febBw2cBKiqdVVv2sHjDTpas38nSTbtZsXk3ry/acHCw+4AObVsd\nDI1eHTLp2bEtvTtm0q19Gzplt07a+aJCDQYzOxu4C0gFHnL3X1V7fhDwKHAC8D/ufnuY9YhIy5ea\nYvTOa0vvvLacNbRzled27Stn5eY9rNi8mxVb9rBi8x5WbtnN7BVbeemjtVR61fUUZGfQtV0burRr\nQ9fc1nTJbU3Xdm1iy3Jb06FtK8xaXniEFgxmlgrcA3wWWA3MMrPJ7r4wrtkW4JvAF8OqQ0TkgKyM\nNIZ0jZ2ZXV1ZeSWrt+5hxZY9rN1Wyrpte1m7vZS120qZu3obU+bvPXguxgEZaSl0bdeGgpwMsjLS\nyGyVRmar1H//mZFKZnoqmRmxx21bpdEm+DM/O4OCnIxmGSxhbjGMAIrdfSmAmT0DXAAcDAZ33whs\nNLNzQ6xDROSQWqWlHJxQsCbuzubdZazdVsrabXtZt72Uddv3smZbKRt37GXttr2U7q9g975y9pRV\nsKesvMoWSE3aZaYzuPOBs8KzGdI1h/6dsshIi3aCwjCDoRuwKu7xamDk4azIzCYAEwB69ux55JWJ\niDSQmZGXlUFeVgbDuh+6vbuzr7ySPWWxsDgQGqVlFewuq2Dd9tLgZL+dPPX+Cvbuj22NpKUY/Ttl\nHQyLA1OJ5DXh+RsJMfjs7g8CD0LsBLeIyxEROSQzo3V6Kq3TU+nQtlWdbSsqneWbY2eGL1wbOyv8\nvU8389cP1xxs0yk7gwmn9OWGsX3DLj3UYFgD9Ih73D1YJiIicVJTjH75WfTLz+K8YV0PLt+yu4yP\nD04hspP87KbZaggzGGYBA8ysD7FAuBwYH+L7iYi0KB3atmJU/zxG9c87dONGFFowuHu5md0ETCF2\nuOoj7r7AzCYGz99vZp2BIiAHqDSzbwND3H1HWHWJiEjdQh1jcPdXgFeqLbs/7v56YruYRESkmdDV\nMkREpAoFg4iIVKFgEBGRKhQMIiJShYJBRESqUDCIiEgVCXfNZzMrAVYc5svzgE2NWE6iSeb+J3Pf\nIbn7r77H9HL3/LoaH5BwwXAkzKyovhfDbomSuf/J3HdI7v6r7w3vu3YliYhIFQoGERGpItmC4cGo\nC4hYMvc/mfsOyd1/9b2BkmqMQUREDi3ZthhEROQQFAwiIlJF0gSDmZ1tZovNrNjMbom6nqZkZsvN\nbJ6ZzTGzoqjrCZuZPWJmG81sftyyDmb2mpl9EvzZPsoaw1JL3281szXB5z/HzD4fZY1hMbMeZvYv\nM1toZgvM7FvB8mT57Gvrf4M//6QYYzCzVGAJ8FlgNbGry41z94WRFtZEzGw5UOjuSXGSj5mdAuwC\nHnf3o4Nlvwa2uPuvgh8G7d39+1HWGYZa+n4rsMvdb4+ytrCZWRegi7t/YGbZwGzgi8A1JMdnX1v/\nL6OBn3+ybDGMAIrdfam7lwHPABdEXJOExN3fAbZUW3wB8Mfg/h+J/YdpcWrpe1Jw93Xu/kFwfyew\nCOhG8nz2tfW/wZIlGLoBq+Ier+Yw/8ISlAOvm9lsM5sQdTERKXD3dcH99UBBlMVE4BtmNjfY1dQi\nd6XEM7PewPHATJLws6/Wf2jg558swZDsxrj7ccA5wNeD3Q1Jy2P7T1v+PtR/uw/oCxwHrAPuiLac\ncJlZFvA88O3q149Phs++hv43+PNPlmBYA/SIe9w9WJYU3H1N8OdG4K/Edq0lmw3BPtgD+2I3RlxP\nk3H3De5e4e6VwCRa8OdvZunEvhT/5O4vBIuT5rOvqf+H8/knSzDMAgaYWR8zawVcDkyOuKYmYWZt\ng4EozKwt8Dlgft2vapEmA1cH968G/hZhLU3qwJdi4EJa6OdvZgY8DCxy99/GPZUUn31t/T+czz8p\njkoCCA7RuhNIBR5x959HXFKTMLO+xLYSANKAp1p6383saeA0YlMObwB+ArwIPAf0JDZt+2Xu3uIG\naWvp+2nEdiM4sBz4Stw+9xbDzMYAU4F5QGWw+L+J7WdPhs++tv6Po4Gff9IEg4iI1E+y7EoSEZF6\nUjCIiEgVCgYREalCwSAiIlUoGEREpAoFgzQbZjY9+LO3mY1v5HX/d03vFRYz+6KZ/Tikdf/3oVs1\neJ3HmNljjb1eSUw6XFWaHTM7Dfh/7n5eA16T5u7ldTy/y92zGqO+etYzHTj/SGe0ralfYfXFzF4H\nrnP3lY29bkks2mKQZsPMdgV3fwWMDeaO/46ZpZrZb8xsVjAR2FeC9qeZ2VQzmwwsDJa9GEwWuODA\nhIFm9iugTbC+P8W/l8X8xszmW+yaFV+KW/dbZvYXM/vYzP4UnFmKmf0qmPN+rpn9x1TGZjYQ2Hcg\nFMzsMTO738yKzGyJmZ0XLK93v+LWXVNfvmxm7wfLHgimmcfMdpnZz83sIzObYWYFwfJLg/5+ZGbv\nxK3+JWKzAkiyc3fddGsWN2JzxkPsTN2X45ZPAH4Y3M8AioA+QbvdQJ+4th2CP9sQO/W/Y/y6a3iv\ni4HXiJ0RXwCsBLoE695ObF6tFOA9YAzQEVjMv7e229XQj2uBO+IePwb8I1jPAGKz+7ZuSL9qqj24\nP5jYF3p68Phe4KrgvgNfCO7/Ou695gHdqtcPjAZeivrfgW7R39LqGyAiEfocMMzMLgke5xL7gi0D\n3nf3ZXFtv2lmFwb3ewTtNtex7jHA0+5eQWyytbeB4cCOYN2rAcxsDtAbmAHsBR42s5eBl2tYZxeg\npNqy5zw2idknZrYUGNTAftXmDOBEYFawQdOGf08SVxZX32xiF6oCmAY8ZmbPAS/8e1VsBLrW4z2l\nhVMwSCIw4BvuPqXKwthYxO5qj88ETnb3PWb2FrFf5odrX9z9CiDN3cvNbASxL+RLgJuA06u9rpTY\nl3y86oN5Tj37dQgG/NHdf1DDc/vd/cD7VhD8f3f3iWY2EjgXmG1mJ7r7ZmJ/V6X1fF9pwTTGIM3R\nTiA77vEU4KvBlMKY2cBgptjqcoGtQSgMAk6Ke27/gddXMxX4UrC/Px84BXi/tsIsNtd9rru/AnwH\nOLaGZouA/tWWXWpmKWbWj9jc+Isb0K/q4vvyBnCJmXUK1tHBzHrV9WIz6+fuM939x8S2bA5MST+Q\nFjrzqjSMthikOZoLVJjZR8T2z99FbDfOB8EAcAk1X57xH8BEM1tE7It3RtxzDwJzzewDd78ibvlf\ngZOBj4j9iv8vd18fBEtNsoG/mVlrYr/Wb66hzTvAHWZmcb/YVxILnBxgorvvNbOH6tmv6qr0xcx+\nCPzTzFKA/cDXic0iWpvfmNmAoP43gr4DfAb4ez3eX1o4Ha4qEgIzu4vYQO7rwfkBL7v7XyIuq1Zm\nlgG8Texqf7Ue9ivJQbuSRMLxCyAz6iIaoCdwi0JBQFsMIiJSjbYYRESkCgWDiIhUoWAQEZEqFAwi\nIlKFgkFERKr4/5l7b56GXNQ6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x104a96dd8>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "parameters = dnn_app_utils_v2.L_layer_model(train_x, train_y, layers_dims, num_iterations = 2500, print_cost = True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在训练集和测试集上进行预测，检测模型准确率，训练集和测试集准确率分别为0.986和0.8：\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "print(‘Train Accuracy’)\n",
    "pred_train = predict(train_x, train_y, parameters)\n",
    "print(‘Test Accuracy’)\n",
    "pred_test = predict(test_x, test_y, parameters)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "从准确率看，相比于第三章的逻辑回归（本章节处理的数据与第三章相同），深层神经网络准确率提高了不少，这是因为深层神经网络提供更多隐藏层，使得神经网络能够拟合更复杂的模型，识别图片准确率更高。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
